Document LLMObs trace sampling controls for Python and Node.js SDKs#37602
Draft
Yun-Kim wants to merge 3 commits into
Draft
Document LLMObs trace sampling controls for Python and Node.js SDKs#37602Yun-Kim wants to merge 3 commits into
Yun-Kim wants to merge 3 commits into
Conversation
Add DD_LLMOBS_SAMPLE_RATE (command-line setup) and the new sample_rate parameter on LLMObs.enable() (in-code setup) to the Agent Observability SDK reference. Python-only; not supported in the Node.js or Java SDKs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Preview links (active after the
|
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Yun-Kim
changed the title
Yun-Kim
commented
Jun 26, 2026
|
|
||
| `DD_LLMOBS_SAMPLE_RATE` | ||
| : optional - _float_ - **default**: `1.0` | ||
| <br />The fraction of traces to submit to Agent Observability, between `0.0` (drop everything) and `1.0` (submit everything). The sampling decision is made on the root span and inherited by all of its child spans, so a trace is always kept or dropped as a whole. The decision is also propagated to downstream services, keeping distributed traces intact. This client-side sampling happens before ingestion and is independent of in-app controls such as [automation rules][2]. It does not affect [APM trace sampling][3], and APM sampling does not drop Agent Observability data. |
Contributor
Author
There was a problem hiding this comment.
Suggested change
| <br />The fraction of traces to submit to Agent Observability, between `0.0` (drop everything) and `1.0` (submit everything). The sampling decision is made on the root span and inherited by all of its child spans, so a trace is always kept or dropped as a whole. The decision is also propagated to downstream services, keeping distributed traces intact. This client-side sampling happens before ingestion and is independent of in-app controls such as [automation rules][2]. It does not affect [APM trace sampling][3], and APM sampling does not drop Agent Observability data. | |
| <br />The fraction of traces to submit to Agent Observability, between `0.0` (drop everything) and `1.0` (submit everything). The sampling decision is made on the root span and inherited by all of its child spans (including downstream services). This sampling is independent of in-app controls such as [automation rules][2] and [APM trace sampling][3]. |
Yun-Kim
commented
Jun 26, 2026
|
|
||
| `DD_LLMOBS_SAMPLE_RATE` | ||
| : optional - _float_ - **default**: `1.0` | ||
| <br />The proportion of traces to sample for Agent Observability, between `0.0` (drop everything) and `1.0` (keep everything). The sampling decision is computed once on the root span, inherited by all of its child spans, and propagated to downstream services, so a distributed trace is kept or dropped as a whole. Spans are always sent to Datadog, and the sampling decision is applied during ingestion. This is independent of in-app controls such as [automation rules][2], and does not affect [APM trace sampling][3]. |
Contributor
Author
There was a problem hiding this comment.
Same suggestion as above (python)
Yun-Kim
commented
Jun 26, 2026
|
|
||
| `sample_rate` | ||
| : optional - _float_ | ||
| <br />The fraction of traces to submit to Agent Observability, between `0.0` (drop everything) and `1.0` (submit everything). The sampling decision is made on the root span and inherited by all of its child spans, so a trace is always kept or dropped as a whole. Values outside the `[0.0, 1.0]` range are ignored. This takes precedence over `DD_LLMOBS_SAMPLE_RATE`; if not provided, this defaults to the value of that environment variable (`1.0` if unset). |
Contributor
Author
There was a problem hiding this comment.
Suggested change
| <br />The fraction of traces to submit to Agent Observability, between `0.0` (drop everything) and `1.0` (submit everything). The sampling decision is made on the root span and inherited by all of its child spans, so a trace is always kept or dropped as a whole. Values outside the `[0.0, 1.0]` range are ignored. This takes precedence over `DD_LLMOBS_SAMPLE_RATE`; if not provided, this defaults to the value of that environment variable (`1.0` if unset). | |
| <br />The fraction of traces to submit to Agent Observability, between `0.0` (drop everything) and `1.0` (submit everything). The sampling decision is made on the root span and inherited by all of its child spans (including downstream services). If unset, this defaults to `DD_LLMOBS_SAMPLE_RATE`, but otherwise takes precedence over the environment variable. |
Yun-Kim
commented
Jun 26, 2026
|
|
||
| `sampleRate` | ||
| : optional - _number_ | ||
| <br />The proportion of traces to sample for Agent Observability, between `0` and `1` (inclusive). The sampling decision is computed once on the root span, inherited by all of its child spans, and propagated to downstream services, so a distributed trace is kept or dropped as a whole. This takes precedence over `DD_LLMOBS_SAMPLE_RATE`; if not provided, it defaults to the value of that environment variable (`1` if unset). |
Contributor
Author
There was a problem hiding this comment.
Same comment for Python inline config
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
XG-xin
approved these changes
Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Documents Python and Node.js LLM Observability (Agent Observability) trace sampling controls in the SDK reference (
instrumentation/sdk.md):Python:
DD_LLMOBS_SAMPLE_RATEenvironment variable — added to the Python command-line setup env-var list (float0.0–1.0, default1.0).sample_rateparameter onLLMObs.enable()— added to the Python in-code setup parameters, following dd-trace-py#18607. Documents precedence (sample_ratearg >DD_LLMOBS_SAMPLE_RATE> default1.0) and that out-of-range values are ignored.Node.js (assumes dd-trace-js#9030 merges):
DD_LLMOBS_SAMPLE_RATEenvironment variable — added to the Node.js command-line setup env-var list (float, default1.0).sampleRateparameter underllmobsintracer.init({ llmobs: { sampleRate } })— added to the Node.js in-code setup parameters. Documents precedence (sampleRate>DD_LLMOBS_SAMPLE_RATE> default1).In the Node.js SDK the keep/drop decision is computed once on the root span, inherited by descendants, and propagated across services via
x-datadog-tags; spans are always shipped and the decision is applied during ingestion.Why
Neither control was documented anywhere in
content/en/llm_observability/. The existing docs only cover server-side/UI sampling (automation rules, annotation queues, datasets, custom judges), so there was no reference for client-side, pre-ingestion trace sampling in the SDK.Scope notes
Claude session: `8bc9e12c-8440-4689-a2dc-e2f1f1ad66ae`
Resume: `claude --resume 8bc9e12c-8440-4689-a2dc-e2f1f1ad66ae`
🤖 Generated with Claude Code